Unclassified  Unlimited  Distribution 


ECBC-TR-XXX 


TEMPORAL-SPECTRAL  DETECTION  IN  LONG  WAVE  INFRARED 

HYPERSPECTRAL  IMAGERY 


Daniel  C.  Heinz 
Charles  E.  Davidson 
Avishai  Ben-David 

Science  and  Technology  Corp. 
500  Edgewood  Rd,  Suite  205, 
Edgewood  MD  21040, 

410  436-7736 

Edgewood  Chemical  Biological 
Center, Aberdeen  Proving 
Ground,  MD  21010, 

410  436-6631 

June  2008 

ADD  Distribution  Statement  -  Unlimited  Distribution 


1 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JUN  2008  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

Temporal- Spectral  Detection  in  Long  Wave  Infrared  Hyperspectral 
Imagery 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Science  and  Technology  Corp,500  Edgewood  Rd,  Suite 

205, Edgewood,MD  ,21040 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

15 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Unclassified  Unlimited  Distribution 


Disclaimer 

The  findings  in  this  report  are  not  to  be  construed  as  an  official  Department  of  the  Army  position 
unless  so  designated  by  other  authorizing  documents. 


2 


Unclassified  Unlimited  Distribution 


Report  Documentation  Page 


3 


Unclassified  Unlimited  Distribution 


TABLE  OF  CONTENTS 


1  Abstract  . 7 

2  Introduction . 7 

3  Temporal- Spectral  Detection  Algorithms . 7 

4  Experimental  Results . 10 

5  Conclusions .  14 

6  References . 14 


4 


Unclassified  Unlimited  Distribution 


FIGURES 

1  Visible  image  with  broadband  IR  overlay  (150x320)  showing  location  of  data 

acquisition  .  10 

2  Detectors  (Eqs.  2-6  &  Eq.  19)  SNR  performance  along  the  plume.  The  black  line 

indicates  the  input  SNR.  The  dark  yellow  line  indicates  predicted  matched  filter 
performance  (Eq.  19)  (Simulation) .  11 

3  Detectors  (Eqs.  7-9)  SNR  performance  along  the  plume.  The  black  line  indicates  the 

input  SNR  (Simulation) .  12 

4  Portion  of  Figure  3  where  SNR  is  lower  (Simulation) .  12 

5  Performance  of  detectors  on  a  release  of  SF6  (Field  Data) .  13 

6  Performance  of  cascaded  detectors  on  a  release  of  SF6  (Field  Data) .  14 

TABLES 

1  SNR  Performance  of  Frame  Averaging .  13 


5 


Unclassified  Unlimited  Distribution 


6 


Unclassified  Unlimited  Distribution 


ABSTRACT 

Ground-based  staring  hyperspectral  chemical  detectors  allow  for  repeated  measurements  through  time 
with  near-perfect  image  registration.  The  problem  with  standard  spectral  based  hyperspectral  detection 
algorithms  is  that  they  do  not  make  effective  use  of  this  temporal  information.  In  this  paper  we  show  that 
significant  improvements  in  detection  performance  for  staring  geometry  can  be  made  by  making  use  of 
statistical  information  obtained  from  previous  samples  and  new  temporal-spectral  detection  algorithms 
are  developed.  These  new  algorithms  have  the  advantage  that  they  limit  detection  to  regions  where  both 
temporally  and  spectrally  significant  events  have  occurred.  We  discuss  the  development  of  these 
algorithms  and  demonstrate  the  performance  of  both  temporal-spectral  and  spectral  detectors  for  detection 
of  gaseous  plumes  using  data  from  the  FIRST  (Field-Portable  Imaging  Radiometric  Spectrometer 
Technology)  passive  long  wave  infrared  (LWIR)  hyperspectral  sensor. 

INTRODUCTION 

Hyperspectral  imagery  is  often  visualized  as  a  three-dimensional  image  cube,  where  two  of  the 
dimensions  are  used  to  indicate  spatial  location  of  an  image  pixel  and  the  third  dimension  specifies  a 
spectral  band.  When  the  hyperspectral  sensor  is  set-up  to  stare  at  a  fixed  location  a  fourth  dimension  of 
time  is  created  as  each  new  cube  is  sampled  in  time.  Therefore,  we  are  dealing  with  ‘hyper’  remote 
sensing  where  we  have  hyperspectral  (many  contiguous  spectral  bands),  hyperspatial  (many  pixels  at 
about  lm  spacing)  and  hypertemporal  (repeated  coverage  every  few  seconds)  imagery  [1],  In  a  ground- 
based  stare-mode  geometry  each  new  cube  will  have  almost  perfect  spatial  registration  with  the  previous 
data  cubes. 

There  are  three  time  scales  inherent  in  a  hyperspectral  sensor  monitoring  for  gaseous  plumes:  electronic 
noise  on  the  MHz  scale,  atmospheric  turbulence  on  the  KHz  scale,  and  long  time  scale  (Hz)  for 
atmospheric  transmission  drift  [2,  3].  Since  the  hyperspectral  sensor  operates  on  the  Hz  scale  the  variance 
induced  on  the  signal  due  to  higher  frequency  (KHz  and  MHz)  will  be  reduced  via  averaging  due  to  the 
independence  of  samples.  We  will  explore  the  amount  of  averaging  required  and  how  it  relates  to  the  third 
time  scale. 

It  will  be  shown  that  by  combining  temporal  and  spectral  information  we  are  able  to  significantly  increase 
the  signal-to-noise  ratio  (SNR).  This  significant  SNR  increase  allows  the  detection  thresholds  to  be 
raised,  which  greatly  reduces  the  false  alarm  rate. 

TEMPORAL-SPECTRAL  DETECTION  ALGORITHMS 

The  temporal-spectral  detection  algorithms  function  to  detect  targets  by  using  statistics  from  time  tt  to 
detect  targets  at  time  t2 .  The  algorithms  are  developed  as  follows.  Let  /  represent  the  number  of  spectral 
bands  and  let  x;  he  an  l  x  1  column  pixel  vector  at  spatial  location  (/,  j)  and  at  sample  time  tx .  To 
simplify  the  following  notation  the  (/,  /  )  spatial  location  information  is  omitted.  In  multivariate  statistical 
analysis  the  Mahalanobis  distance  squared  of  a  test  pixel  spectrum  from  the  mean  of  the  background  class 
is  given  by 
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AD 
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and  in  the  literature  is  known  as  an  anomaly  detector  (AD)  [4,5].  In  Eq.  1  the  mean,  p  and  covariance, 

C,  are  measured  only  once  at  a  time  t0  when  it  is  assumed  that  the  target  is  not  present  in  the  scene.  This 

method  assumes  that  the  mean  and  covariance  are  stationary.  There  are  many  ways  to  calculate  the  mean 
and  covariance  (e.g.  image  segmentation),  in  this  paper  we  use  the  global  mean  and  covariance. 

By  turning  Eq.  1  into  a  ratio  and  making  use  of  both  past  and  present  statistics  we  can  form  two  temporal- 
spectral  detectors.  The  first,  referred  to  as  the  temporal-spectral  anomaly  detector  ( TSAD ),  is  given  as 
follows 


_(u2-^2)rC''(xf2-p  ) 


(2) 


The  second,  referred  to  as  the  temporal-spectral  covariance  detector  ( TSCD  ),  is  given  as  follows 


(3) 


Eqs.  2  and  3  are  ratios  of  anomaly  detectors,  in  order  for  the  TSAD  to  improve  detection  performance 
the  anomaly  (e.g.  gaseous  plume)  is  assumed  not  to  be  present  at  pixel  location  (/,  j)  at  time  tl  or  at  least 
that  the  concentration  of  target  gas  has  increased  from  time  tl  to  time  t2  at  pixel  location  (/,  /  ) .  Both  the 
TSAD  and  TSCD  assume  that  the  covariance  matrix  formed  at  time  tx  is  free  from  a  temporal  signal  or 
when  the  gas  concentration  has  increased  from  time  tx  to  time  t2  to  have  less  temporal  signal  than  the 
covariance  matrix  formed  at  time  t2 .  The  TSAD  further  assumes  that  the  plume  is  either  present  in  only 
a  portion  of  the  scene  or  non-uniform.  Under  these  assumptions  both  the  ratios  in  Eqs.  2  and  3  will  be 
larger  than  one  for  any  pixel  vector  x;  that  contains  an  anomalous  temporal  signal.  Elowever,  if  the  gas 

concentration  is  determined  to  be  decreasing  when  the  system  is  started  then  the  ratios  in  both  Eqs.  2  and 
3  can  be  inverted  in  order  to  improve  detection  performance. 


Eqs.  2  and  3  are  easily  coupled  with  any  standard  detector  such  as  the  matched  filter  (MF)  or  adaptive 
coherence/cosine  estimator  (ACE)  detector  [5],  The  matched  filter  is  given  by  Eq.  4 


D,,r, 


(u 


trc:t 


(4) 


where  t  is  the  target  vector  of  interest.  Like  the  AD,  the  mean  and  covariance  are  measured  only  once  at 
a  time  t0 .  Again,  this  assumes  that  the  mean  and  covariance  are  stationary.  Another  variation  of  the 
matched  filter  given  by 


Dmfu  ~ 


Ucr't 


(5) 


uses  data  only  at  the  present  time.  This  method  does  not  require  the  stationarity  of  Eq.  4,  but  its 
performance  will  be  reduced  by  the  presence  of  signal  at  time  t2  due  to  whitening.  By  incoiporating  the 
use  of  covariance  matrix  at  time  tx  into  Eq.  5  we  get  another  version  of  the  matched  filter  that  uses  the 
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covariance  matrix  formed  at  time  tl  when  the  scene  is  potentially  free  from  target  signal  or  has  less  target 
signal  than  the  covariance  matrix  formed  at  time  t2 


D 


MFtx 


UC-'t 


(6) 


This  method  assumes  stationarity  over  the  short  period  from  tl  to  t2 .  By  coupling  the  TSAD  and/or 
TSCD  detectors  with  the  matched  detector  the  following  three  temporal-spectral  matched  filter  ( TSMF  ) 
detectors  are  formed. 


^TSMFAD  =  DMFtx  X  DTSAD 

^tsMFCD  =  D MFt ,  X  DTSCd 
DtsMF  =  DMFtx  X  DTSAD  X  DTSCD 


(7) 

(8) 
(9) 


The  advantage  of  the  last  3  detectors  (Equations  7-9)  is  that  they  combine  temporal  knowledge  with 
spectral  knowledge.  Therefore,  they  only  alarm  when  a  pixel  vector  contains  both  a  temporally  anomalous 
signal  which  is  a  good  match  spectrally  with  the  target  vector. 

For  the  matched  filter  SNR  performance  can  be  characterized  once  the  inverse  covariance  matrix  is 
known.  If  we  let 


y0~/V(0,C)  (10) 

yj~JV(t,C)  (11) 


where  y0  is  the  signal  model  under  hypothesis  H0  and  signal  model  under  hypothesis  Hl,  with  t 
representing  the  target  signal  vector  and  C  the  covariance  matrix.  We  can  then  whiten  y0  and  y ,  by 

_i 

multiplying  by  C  2  .  If  we  let 

_i 

zo  =C  2y„ 

J_ 

Z1  =c  2yj 

Then 

zo  ~  7V(0,l) 

z,  ~  /V(s,l) 

where, 

_i 

s  =  C  2t  (16) 

The  unnormalized  matched  filter  is  given  by: 

M  =  zrs  (17) 


(14) 

(15) 


(12) 

(13) 


9 


Unclassified  Unlimited  Distribution 


then 


M 


f  N 

(o,s's) 

under  H0  \ 

[iv(s's,s's 

)  under  Hl  J 

(18) 


SNR  can  be  defined  as  the  mean  score  under  Hx  divided  by  the  standard  deviation  of  the  score  under 
H0  ;  therefore,  SNR  for  the  matched  filter  is  simply 


£/Vi?=Vs7s.  (19) 

EXPERIMENTAL  RESULTS 

The  Data  used  in  the  following  experiments  are  from  FIRST  hyperspectral  sensor  [6],  The  data  was 
collected  in  2006  at  Dugway,  UT.  Three  FIRST  sensors  with  focal  plane  array  up  to  256  x  320  were  used 
to  capture  data.  The  sensors  were  setup  to  stare  at  a  fixed  location  where  the  simulant  would  be  released. 
Frames  were  captured  approximately  every  6  seconds.  Visible  imagery  was  collected  using  a  co-aligned 
camera.  Figure  1  shows  the  visible  image  with  broadband  IR  overlay  (150  x  320). 


Figure  1.  Visible  image  with  broadband  IR  overlay  (150x320)  showing  location  of  data  acquisition. 


Since  there  are  8  methods  to  compare  (Equations  2-9)  we  started  with  only  Equations  2-6.  We  began  by 
examining  the  performance  of  each  algorithm  versus  SNR.  A  simulation  was  conducted  by  inserting  SF6 
from  our  library  spectra  into  row  64  of  a  data  cube  free  of  target  signal  using  half  the  focal  plane  array 
(128  x  320).  The  signal  was  set  to  have  a  SNR  of  approximately  45  (this  corresponds  to  a  concentration 
path  length  of  approximately  135  mg/m2  and  delta  temperature  of  2  degrees)  in  column  1  of  row  64  and 
linearly  decay  to  0  in  column  320  of  row  64.  The  signal  to  noise  was  calculated  by  taking  the  norm  of  the 
inserted  SF6  signal  divided  by  the  standard  deviation  of  the  noise  measured  at  each  pixel  in  row  64.  The 
input  SNR  can  be  used  as  a  measure  for  the  matched  filter  performance;  for  instance,  if  the  inverse 
covariance  matrix  is  equal  to  the  identity  matrix  then  the  matched  filter  SNR  will  be  about  equal  to  the 
measured  input  SNR  (Eq.  19).  However  if  the  target  vector  is  whitened  by  the  inverse  covariance  matrix 
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then  the  matched  filter  performance  will  be  poor.  This  is  the  main  advantage  of  the  temporal-spectral 
approach  in  which  earlier  covariance  matrices  without  target  are  used  to  enhance  SNR  performance. 


Column 

Figure  2.  Detectors  (Eqs.  2-6  &  Eq.  19)  SNR  performance  along  the  plume.  The  black  line  indicates  the 
input  SNR.  The  dark  yellow  line  indicates  predicted  matched  filter  performance  (Eq.  19)  (Simulation). 

Figure  2  contains  a  plot  of  the  detectors  (Eqs.  2-6  &  Eq.  19)  SNR  performance  where  the  noise  was 
calculated  by  taking  the  standard  deviation  of  all  pixels  except  for  those  in  row  64.  The  two  matched 
filters  MFt  and  MFt  had  almost  identical  performance.  Their  performance  was  mostly  equal  to  the 

predicted  matched  filter  SNR  performance  of  Eq.  19.  The  matched  filter,  MF  of  Eq.  4  performed 

slightly  worse  than  the  other  two  matched  filters  since  the  data  was  not  completely  stationary.  The  TSAD 
(Eq.  2)  had  the  worst  performance  of  all  the  detectors  at  low  SNR  values,  but  did  the  best  at  higher  SNR 
values.  The  TSCD  (Eq.  3)  performed  similar  to  the  matched  filters. 

The  three  additional  detectors  (Eqs.  7-9)  were  formed  by  coupling  the  TSAD  and/or  TSCD  detectors 
with  the  matched  detector.  Figure  3  shows  the  results  for  these  detectors  using  the  same  data  and  input 
signal  as  in  Figure  2.  As  we  can  see,  at  the  higher  SNR  values  the  TSMF  performed  extremely  well. 
Figure  4  shows  the  region  in  Figure  3  from  columns  160  to  column  320.  Comparing  Figure  2  and  4 
we  see  that  performance  the  TSMF  AD  and  TSMF  of  Figure  4  are  closer  to  the  input  SNR  than  the 
matched  filters  of  Figure  2  at  low  input  SNR  values  and  start  to  have  a  significant  improvement  in 
performance  for  input  SNR  values  of  12  and  greater. 

In  the  following  we  analyze  the  performance  of  the  detectors  for  field  data  with  a  release  of  SF6  using 
library  spectra  as  the  target  vector.  In  order  to  determine  if  frame  averaging  would  help  reduce  the  noise 
we  examined  the  global  mean  over  100  frames  of  data  without  any  target  signal  present.  Window  sizes 
from  1  to  7  frames  over  the  1 00  frame  sequence  where  used  to  calculate  the  mean  standard  deviation  of 
global  mean.  The  mean  standard  deviation  followed  the  expected  square-root  of  the  number  of  frames 
reduction  for  window  size  up  to  about  5  frames.  Therefore,  the  mean  is  stationary  for  about  five  frames 
(30  seconds).  This  means  that  the  atmospheric  transmission  drift  on  the  Flz  scale  starts  to  become 
significant  after  about  30  seconds.  In  [1]  it  was  shown  that  atmospheric  drift  was  significant  for  periods  as 
shorts  as  5  seconds. 
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Figure  3.  Detectors  (Eqs.  7-9)  SNR  performance  along  the  plume.  The  black  line  indicates  the  input  SNR 
(Simulation). 


Column 

Figure  4.  Portion  of  Figure  3  where  SNR  is  lower  (Simulation). 


The  second  factor  in  determining  how  many  frames  to  average  is  the  signal.  If  the  signal  remains  present 
in  a  pixel  for  multiple  frames  then  it  is  beneficial  to  average  and  reduce  the  noise  as  stated  above.  The 
signal  movement  will  be  dependent  on  the  size  of  the  plume  and  the  wind  speed  and  direction.  By 
examining  the  plume  in  the  field  data  it  was  determined  that  the  wind  speed  was  approximately  2  m/s  in  a 
direction  perpendicular  to  the  sensor  and  the  plume  was  about  72  meters  across  as  viewed  by  the  sensor 
and  at  a  distance  such  that  each  pixel  is  lm  x  lm.  Consequently,  the  plume  will  remain  in  a  pixel  for 


12 


Unclassified  Unlimited  Distribution 


about  36  seconds  or  six  frames.  However,  the  plume  was  not  of  uniform  intensity  with  the  brightest 
portion  being  near  the  center  of  the  plume  and  the  dimmest  portion  at  the  outer  radii. 

In  order  to  determine  if  frame  averaging  is  useful,  a  subset  of  the  data,  which  contained  signal  with  low 
SNR,  consisting  of  rows  1  through  70  and  columns  1  through  320  was  used.  The  weak  signal  in  row  65 
was  used  to  measure  performance.  Table  I  contains  results  for  the  SNR  performance  of  the  TSMF  with 
from  1  to  5  frames  of  averaging. 


Table  1.  SNR  Performance  of  Frame  Averaging 


Frames  Averaged 

SNR  Performance 

1 

24.1 

2 

31.4 

3 

43.2 

4 

33.2 

5 

23.8 

In  this  case,  three  frames  of  averaging  had  the  best  performance.  Figure  5  contains  results  for  Equations  2, 
3,  and  6-9  with  the  noise  normalized  to  that  of  the  TSMF  detector  using  three  frames  of  averaging  for 
both  tx  and  t2 . 


Figure  5.  Performance  of  detectors  on  a  release  of  SF6  (Field  Data). 

As  we  can  see,  TSMF  performed  the  best.  TSCD  performed  second  best,  and  performance  of  the 
TSMFCD  was  worse  than  TSCD  .  Based  on  these  results  we  may  want  to  create  a  temporal  spectral 
detector  by  coupling  Eqs.  2  and  3.  However,  Eqs.  2  and  3  are  anomaly  detectors  and  will  alarm  to  any 
anomalous  gaseous  plume,  but  the  matched  filter  will  only  alarm  to  target  vector  like  plumes. 

If  the  plume  is  present  in  a  pixel  for  more  than  one  frame  we  can  improve  performance  further  by 
cascading  the  detector  outputs.  For  instance,  if  we  repeat  the  experiment  of  Figure  5,  but  multiply  the 
output  of  two  consecutive  detector  outputs  we  get  the  results  of  Figure  6.  This  results  in  a  significant 
SNR  performance  improvement.  These  high  SNR  values  allow  the  detection  thresholds  to  be  raised  which 
will  result  in  a  significant  drop  in  false  alarms.  Even  if  the  plume  is  not  present  in  a  single  pixel  during 
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two  consecutive  frames  we  could  use  a  more  sophisticated  algorithm  which  searches  the  FOV  to  align  the 
plume  from  one  frame  to  the  next,  provided  there  is  adequate  SNR  in  each  frame. 


Figure  6.  Performance  of  cascaded  detectors  on  a  release  of  SF6  (Field  Data). 


CONCLUSIONS 

This  paper  presented  a  set  of  temporal-spectral  detectors  for  target  detection  in  hyperspectral  imagery. 
This  set  is  not  complete  since  the  TSAD  and  TSCD  detectors  can  be  coupled  with  any  standard  detector. 
The  advantage  of  the  coupled  detectors  is  that  they  combine  temporal  knowledge  with  spectral  knowledge 
and  only  alarm  to  temporally  anomalous  signals  which  are  a  good  match  spectrally  with  the  target  vector. 
The  significant  increase  in  SNR  from  this  approach  allows  the  detection  thresholds  to  be  raised  and  the 
false  alarm  rate  to  be  greatly  reduced.  The  FIRST  data  experiments  demonstrated  the  effectiveness  of  this 
approach. 
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